All simulations were based on pytorch [5]. For the nonlinear neuroscience tasks, we applied the gradient descent method "Adam" [4] to the recurrent weights W as well as to the input and output vectors mi, wi. We checked that our results did not depend qualitatively on the choice of the "Adam" algorithm over plain gradient descent; however, training converged more easily for this choice of algorithm. We also checked that restricting training to W only (as for the simple model) did not alter our results qualitatively (although, with this restriction, training on the Romo task for small values of g did not converge). Code for reproducing our results can be found on https://github.com/frschu/neurips_

In digital health, the strategy of allocating a limited treatment budget across available risk times is crucial to reduce user fatigue. This strategy, however, encounters a significant obstacle due to the unknown actual number of risk times, a factor not adequately addressed by existing methods lacking theoretical guarantees. This paper introduces, for the first time, the online uniform risk times sampling problem within the approximation algorithm framework. We propose two online approximation algorithms for this problem, one with and one without learning augmentation, and provide rigorous theoretical performance guarantees for them using competitive ratio analysis. We assess the performance of our algorithms using both synthetic experiments and a real-world case study on HeartSteps mobile applications.

A model and expansion plan have been developed to optimally determine microgrid designs as they evolve to dynamically react to changing conditions and to exploit energy storage capabilities. In the wake of the highly electrified future ahead of us, the role of energy storage is crucial wherever distributed generation is abundant, such as microgrid settings. Given the variety of storage options that are recently becoming more economical, determining which type of storage technology to invest in, along with the appropriate timing and capacity becomes a critical research question. In problems where the investment timing is of high priority, like this one, developing analytical and systematic frameworks for rigorously considering these issues is indispensable. From a business perspective, these strategic frameworks will aim to optimize the process of investment planning, by leveraging novel approaches and by capturing all the problem details that traditional approaches are unable to. Reinforcement learning algorithms have recently proven to be successful in problems where sequential decision-making is inherent. In the operations planning area, these algorithms are already used but mostly in short-term problems with well-defined constraints and low levels of uncertainty modeling. On the contrary, in this work, we expand and tailor these techniques to long-term investment planning by utilizing model-free approaches, like the Q-learning algorithm, combined with simulation-based models. We find that specific types of energy storage units, including the vanadium-redox battery, can be expected to be at the core of the future microgrid applications, and therefore, require further attention. Another key finding is that the optimal storage capacity threshold for a system depends heavily on the price movements of the available storage units in the market.